Combining Multimodal Sensory Input for Spatial Learning
نویسندگان
چکیده
Motivation: Understand multisensory integration in spatial representations. • Hippocampal place cells form multimodal representation of space. • Cells in superior colliculus show multimodal enhancement. • Relevance of a mode depends on environmental conditions. • We propose a model on how to weigh modalities using a gating network. • Place code quality in varying conditions drives learning of gater parameters. • Implementation / validation of model on a mobile robot platform.
منابع مشابه
Multimodal sensory integration and concurrent navigation strategies for spatial cognition in real and artificial organisms.
Flexible spatial behavior requires the ability to orchestrate the interaction of multiple parallel processes. At the sensory level, multimodal inputs must be combined to produce a robust description of the spatiotemporal properties of the environment. At the action-selection level, multiple concurrent navigation policies must be dynamically weighted in order to adopt the strategy that is the mo...
متن کاملCrossmodal identification Gemma
senses, each tuned to a distinct form of energy and providing a unique window through which to experience the environment. The possession of multiple sensory systems provides considerable behavioural flexibility since input from one modality can substitute for another under circumstances of specific sensory deprivation. In darkness, for example, auditory and tactile cues might supplant visual i...
متن کاملCombining User Modeling and Machine Learning to Predict Users' Multimodal Integration Patterns
Temporal as well as semantic constraints on fusion are at the heart of multimodal system processing. The goal of the present work is to develop useradaptive temporal thresholds with improved performance characteristics over state-of-the-art fixed ones, which can be accomplished by leveraging both empirical user modeling and machine learning techniques to handle the large individual differences ...
متن کاملGrounded spoken language acquisition: experiments in word learning
| Language is grounded in sensory-motor experience. Grounding connects concepts to the physical world enabling humans to acquire and use words and sentences in context. Currently most machines which process language are not grounded. Instead, semantic representations are abstract, pre-speci ed, and have meaning only when interpreted by humans. We are interested in developing computational syste...
متن کاملApprentissage spatial de corrélations multimodales par des mécanismes d'inspiration corticale. (Spatial learning of multimodal correlations in a cortically inspired way)
This thesis focuses on unifying multiple modal data flows that may be provided by sensors of anagent. This unification, inspired by psychological experiments like the ventriloquist effect, is based ondetecting correlations which are defined as temporally recurrent spatial patterns that appear in the inputflows. Learning of the input flow correlations space consists on sampling this ...
متن کامل